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Objectives: This study aims to investigate the suitability of electronic health record (EHR) systems in Korea for global certi- 
fication and to propose functions for future global systems by comparing and analyzing the certification criteria for Certifi- 
cation Commission for Health Information Technology (CCHIT) Certified Ambulatory EHR with BESTCare, which is the 
EHR system at Seoul National University Bundang hospital. Methods: Domain expert groups were formed to analyze the in- 
clusion of BESTCare functions and the types of differences for each of the CCHIT Certified 201 1 Ambulatory EHR Certifica- 
tion Criteria. The types of differences were divided into differences in functions (F), differences in business processes (B), and 
differences in government policies (P). Results: Generally the criteria that showed differences in functions pertained to the 
connection between the diagnosis/problem list and order, the alert and warning functions for medication-diagnosis interac- 
tions, and the reminder/instruction/notification messages related to the patient's immunization status; these absent functions 
were enhanced clinical decision support system (CDSS) functions related to patient safety and healthcare quality. Differences 
in government policies were found in the pharmacy's electronic prescription functions, while differences in business pro- 
cesses were found in the functions constrained by the local workflow or internal policy, which require some customization. 
Conclusions: Functions that differed between the CCHIT certification criteria and the BESTCare system in this study should 
be considered when developing a global EHR system. Such a system will need to be easily customizable to adapt to various 
government policies and local business processes. These functions should be considered when developing a global EHR sys- 
tem certified by CCHIT in the future. 
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I. Introduction 

The term "Healthcare Information System Certification" 
refers to a system for follow-up management products, 
services, and compliance with standard specifications for 
healthcare information systems (e.g., order communication 
system, electronic medical record system, picture archiving 
and communication system) [1,2]. A certification system is 
designed to build the foundation to maintain a health infor- 
mation system that is efficient, ensures interoperability, and 
secures the reliability of products [3] . 
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With the recent passage of the economic stimulus bill in the 
United States, interest in electronic health record (EHR) sys- 
tems for the healthcare industry has increased worldwide [3], 
This bill presents a program to provide incentives to clinics 
and hospitals that adopt EHR systems, ensuring interoper- 
ability across various healthcare standards. Numerous EHR 
systems have been developed as a result of such incentive 
programs, and many clinics and hospitals are adopting 
EHR systems certified by the Certification Commission for 
Health Information Technology (CCHIT). In response to 
the changes in the US healthcare industry, some companies 
in South Korea are trying to enter the U.S. healthcare market 
on the basis of their experience in establishing EHR systems 
for hospitals in South Korea [4,5]. 

There are five lists of functional criteria for the certification 
of healthcare information systems, including COPIC (222 
elements), Bureau of Primary Health Care (BPHC, 266 ele- 
ments), Health Level Seven (HL7, 138 elements), CCHIT 
(315 elements), and Practice Fusion (PF, 252 elements) [6- 
10]. The HL7, CCHIT, and PF EHR functionality lists each 
have three major categories. HL7's three major categories are 
direct care and supportive and information infrastructure; 
CCHIT's categories are functionality, interoperability, and 
security and reliability; and PF's categories are features and 
functions, interoperability, and security and reliability [11]. 
This study examined and analyzed CCHIT, which contains 
the most elements. Given that the U.S. government urged 
the extensive use of health information technology (HIT) 
and the routine use of EHR, the U.S. HIT associations have 
established CCHIT as a voluntary private organization 
to certify HIT products. Currently, CCHIT operates four 
working groups (WGs). In addition, CCHIT develops EHR 
certification criteria and procedures, as well as certification 
criteria for the network connecting healthcare information 
systems. By dividing the target areas for the development of 
certification criteria into the ambulatory, inpatient, emer- 
gency department, and enterprise domains, CCHIT also de- 
velops and utilizes the certification criteria for functionality, 
interoperability, and security [4,5,12]. The certification cri- 
teria for functionality are developed to assess functionality 
related to the creation and management of EHR as well as to 
digitalize administrative tasks. The certification criteria for 
interoperability are developed to assess interoperability re- 
lated to information exchange and connection between dif- 
ferent EHR systems, and the certification criteria for security 
are developed to assess the security measures that protect 
patient information in the EHR system [2] . The EHR (BEST- 
Care) at Seoul National University Bundang Hospital is the 
first system outside of North America to receive a stage 7 
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certification from Healthcare Information and Management 
Systems Society (HIMSS) analytics EMR adoption model 
[13]; it is a full EHR system implemented with standard ter- 
minology, standard practice guidelines, a real-time medica- 
tion system using the radio frequency identification (RFID) 
system for drugs, the clinical decision support system (CDSS) 
for antibiotics, blood transfusions and diagnostic tests, and 
the ability to exchange electronic medical information be- 
tween clinics and hospitals using the international standard. 

By comparing and analyzing the certification criteria for 
CCHIT Certified Ambulatory EHR in the U.S. and the 
functionality of BESTCare, this study aims to investigate the 
suitability of EHR systems in Korea to meet the global cer- 
tification requirements as well as to examine the necessary 
functions in detail. Such study is needed to propose func- 
tions for future global systems. 

II. Methods 

For this study, domain expert groups were formed to review 
all criteria presented in the CCHIT Certified 2011 Ambula- 
tory EHR Certification Criteria April 7, 2010 [4,5] . 

The expert groups were composed of medical staff (2 doc- 
tors, 6 nurses), system developers (5 people), and security 
specialists (1 system security specialist, 1 security manager). 
Based on expert group's review, all experts get together at 
the same spot, shared each of all functions, checked with the 
test script and classified inclusion or exclusion of BESTCare 
functionality according whether the suggested test script can 
be executed or not. For functions that were not included, 
the deficiencies were classified as functions (F), business 
processes (B), and government policies (P). Differences in 
functions (F) indicate the absence of a function itself, while 
differences in business processes (B) refer to functions that 
are limited or different because of policies that are internal to 
the hospital system or workflow. Differences in government 
policies (P) refer to functions that are restricted by Korean 
domestic medical law, i.e., these policies cannot be imple- 
mented because of the differences in policy between the US. 
and South Korea. 

III. Results 

In order to check how well BESTCare Ambulatory HER 
functionality suits certification standards which the inter- 
national standard suggests, we investigated whether each 
CCHIT Ambulatory EHR function was included or not in 
BESTCare Ambulatory EHR system, 69.9% (200) of 286 
CCHIT criteria were mapped. Table 1 shows the results of 
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mapping by category (functionality, interoperability, and 
security). For each category (functionality, 229 criteria; 
interoperability, 8 criteria; and security, 49 criteria), the 
mapping rates were 67.7% (155), 12.5% (1), and 89.8% (44), 
respectively. 



1. Functionality Category 

1) Differences in functions 

Table 2 shows the functionality criteria in which differences 
were found between the U.S. CCHIT Ambulatory EHR cer- 



Table 1. Inclusion of criteria in the BESTCare system for each of the CCHIT category 



Category 


No. of criteria 


Yes 






No 




F 


B 


P 


Total 


Functionality 


229 


155 (67.7) 


64 (27.9) 


2 (0.9) 


8 (3.5) 


74 (32.3) 


Interoperability 


8 


1 (12.5) 


2 (25.0) 


0(0) 


5 (62.5) 


7 (87.5) 


Security 


49 


43 (89.8) 


5 (10.2) 


0(0) 


0(0) 


5 (10.2) 


Total 


286 


200 (69.9) 


71 (24.8) 


2 (0.7) 


13 (4.5) 


86 (30.1) 



Values are presented as number (%). F, B, and P mean differences in functions, local business processes, and government policies, 
respectively. 

CCHIT: Certification Commission for Health Information Technology. 



Table 2. Differences in functionality 



Function group 


CCHIT category 


No. of criteria 


Compiling lists 


Allergies and adverse reactions 


3 




Medications 


3 




Problem management 


2 




Orders for diagnostic tests 


1 




Orders for medications 


5 


Creating orders 


Orders and referral management 


1 


Managing workflow 


Clinical task assignment and routing 


5 




Inter-provider communication 


2 




Immunization administration 


1 


Organizing patient data 


Clinical documents and notes 


4 




Patient advance directives 


3 




Patient demographics 


2 


Receiving and displaying 


Capture of external clinical documents 


1 


information 


Health record output 


1 




Patient record identification and maintenance 


1 




Result management 


3 




Report generation 


4 




Health-record summaries 


1 


Supporting decisions 


Notifications and reminders for disease management, preventive 
services, and wellness 


8 




Alerts for disease management, preventive services, and wellness 


10 




Support for drug interactions 


4 


Total 




64 



CCHIT: Certification Commission for Health Information Technology. 
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tification criteria and BESTCare. 

A "compiling list" group is a set of criteria related to prob- 
lems, medication, allergies, and adverse reactions. This term 
refers to management functions under the categories "Aller- 
gies and adverse reactions," "Medications," "Problems," "Di- 
agnostic tests to order," and Medications to order." A detailed 
examination of this group has shown that the "Allergies and 
adverse reactions" category is missing functions that can 
explicitly record whether there was an allergy review (the 
ID used and the date of review) and functions that display 
patient allergy lists and the date of information entry for al- 
lergies. The "Medications" list lacked functions to explicitly 
display that the patient has no prescriptions and functions to 
enter and check new prescriptions. Additionally, a function 
is needed to record why certain drugs may be excluded from 
the current medication list of medications. The "Problem 
management" list must be able to connect orders and pre- 
scriptions for medication for one or more problems/diagno- 
ses when the patient information is protected. Additionally, 
the category "Orders for diagnostic tests" lacked a func- 
tionality that could capture the details of how the tests were 
related to the diagnosis. These deficiencies emerged because 
connections between the current principal diagnosis and the 
medications prescribed were not managed on the database. 

The following alerts and practical medication-identification 
functions were required to be added to the category "Orders 
for medications": 1) display a dose calculator for patient-spe- 
cific dosing based on weight; 2) alert the user if the drug-in- 
teraction information is outdated; 3) add reminders for nec- 
essary follow-up tests based on the medication prescribed; 4) 
alert the user when a new medication is prescribed/ ordered 
that no drug-interaction, allergy, and formulary checking 
will be performed against an uncoded or free-text medica- 
tion; and 5) identify medication samples dispensed, includ- 
ing the lot number and expiration date. 

The category "Orders and referral management in the "Cre- 
ating orders" group also emphasized the connection between 
the problem/ diagnosis and the order/medication by requir- 
ing the addition of functions related to connections between 
the problem/ diagnosis and the prescription ordered. 

The "Managing workflow" list has some communication 
functions within the EHR system, but the following nec- 
essary functions were absent: 1) clinical task assignment 
and routing: create and assign tasks by user or user role; 
designate a task as completed; present a list of tasks by user 
or user role; re-assign and route tasks from one user to an- 
other user; remove a task without completing the task; 2) 
inter-provider communication: document verbal/telephone 
communication in the patient record; support messaging 
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between users; and 3) manage immunizations: capture, in a 
discrete field, an allergy/adverse reaction to a specific immu- 
nization. 

For the criteria "Organizing patient data," a detailed imple- 
mentation of the following functions was required for each 
category: 1) manage clinical documents and notes: filter, 
search or order notes by associated diagnosis within a pa- 
tient record; display patient notes in a manner that distin- 
guishes them from other content in the system; graph height 
and weight over time; display modified notes in full, includ- 
ing both the original content and any changes, corrections, 
clarifications, addenda, etc.; 2) manage patient advance 
directives: indicate that a patient has completed advance 
directive(s); indicate the type of advance directives; indicate 
when advance directives were last reviewed; and 3) man- 
aging patient demographics: maintain and make available 
historic information for demographic data including prior 
names, addresses, phone numbers, and email addresses; 
search information by patient's first and last name. 

The criteria for receiving and displaying information are 
related to test results, patient consent, authorizations, and 
clinical documents from outside the practice. These criteria 
include the categories "Capture external clinical documents," 
"Health-record output," "Identification and maintenance of 
patient records," "Result management," "Report generation," 
and "Health-record summaries." The missing functions 
are as follows: 1) retrieve indexed, scanned documents by 
document type and date; 2) define one or more reports as 
the formal health record for disclosure purposes; 3) merge 
information from two patient records into a single record; 

4) indicate normal and abnormal results based on data from 
the original data source; notify and forward a result; and 

5) produce reports based on the absence of specific clinical 
data; save report parameters to generate subsequent reports; 
modify one or more parameters of a saved report specifica- 
tion when generating a report using that specification. 

The category "Decision support" is a criteria group related 
to alerts and reminders for disease management, preventive 
services, and wellness. Its missing functions are summarized 
below: 1) providing notifications and reminders related to 
immunizations and identifying functions for disease man- 
agement as well as for preventive and wellness services based 
on patient demographic and clinical data; 2) providing in- 
dividualized alerts and functions for up dating/ overriding 
guidelines for disease management or prevention, modifying 
rules for guideline-related alerts, and establishing criteria for 
disease management as well as for preventive and wellness 
services based on patient demographic and clinical data; and 
3) support for drug-interaction alerts: set the severity level 
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at which drug interaction warnings should be displayed; 
display, on demand, potential drug- diagnosis interactions; 
check for a potential interaction between newly documented 
allergies and the patient's current medications. 

For example, the following test script is required during a 
certification test for immunization. 

[Test Script 4.81] Generate reminder letters for patients who 
are due or overdue for a DTaP immunization booster: either 
automatically generate a letter to a patient (either Emily Jones 
or Will Haynes) that automatically includes content specifying 
what services are due or automatically generate a letter to all 
patients who are due for a specified service (DTaP). 

2) Differences in business processes 

Criteria that showed differences in the business process 
were found in the "Problem management" and "Result man- 
agement" categories (Table 3). Missing elements were not 



implemented in the BESTCare system because functions to 
capture, maintain, and display the free-text comments asso- 
ciated with a given problem/ diagnosis are currently restrict- 
ed by hospital policy. As an authorized user in the current 
system can verify a result at any time without forwarding 
information, such a function was found to be unnecessary. 

3) Differences in government policies 

The criteria in which differences arise from government 
policies are shown in Table 4. For example, in South Korea, 
the national healthcare policy requires verification of the 
patient's medical eligibility; in the U.S., a different evaluation 
system is used. Other differences lie in the transmission of 
electronic prescription to pharmacies, the ability to verify a 
patient's eligibility for electronic prescription coverage, and 
the differences in the ability to refill a prescription without 
re-entering data. 



Table 3. Difference in business processes among functionality criteria 



Category/Criteria 


No. of criteria 


Manage problem list 


1 


Capture, maintain, and display free-text comments associated with the problem/diagnosis 


Manage results 


1 


Forward a result to other users 



Total 2 



Table 4. Differences in government policies among functionality criteria 



Category/Criteria 


No. of criteria 


Eligibility verification and determination of coverage 


1 


Display medical eligibility obtained from patient's insurance carrier 


Rules-driven financial and administrative coding assistance 


1 


Provide assistance with selecting an appropriate CPT evaluation and management billing code based 




on codified clinical information in the encounter 




Order medication 


5 


Allow the user to configure prescriptions to incorporate fixed text according to the user's specifications. 


Receive and display information received through electronic prescription eligibility checking 


Reorder a prior prescription without re-entering data (e.g., administration schedule, quantity) 


Ability to print and electronically fax prescriptions. 


Ability to re-print and re-fax prescriptions. 


Pharmacy communication 


1 


Ability to provide electronic communication between prescribers and pharmacies or other intended 




recipients of the medication order 




Total 8 


CPT: current procedural terminology. 
Vol. 18 • No. 1 • March 2012 
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2. Interoperability Category 

Table 5 shows the results of mapping BESTCare over the 
CCHIT interoperability criteria. 

CCHIT laboratories must be able to receive lab results us- 
ing the Health Level 7 (HL7) v2.5.1 message standard. This 
function has also become available in the BESTCare system 
following the increased awareness of international standards 
in Korea. 

Most functions related to the "Medications/ePrescribing" 
category were connected with pharmacies, such as sending 
electronic prescriptions or requesting refills, as seen in Table 
5. Although these functions are limited by government poli- 
cies in Korea, they can be considered essential to fulfilling 
the international requirements. 

The "Clinical documentation" category requires functions 
to generate, file, and display patient summary documents 
in the Healthcare Information Technology Standards Panel 
(HITSP) C32/ Continuity of Care Document (CCD) stan- 
dard format, but because BESTCare can only generate, send, 
and display referral notes or consultation notes in the HL7 
clinical document architecture (CDA), the functions were 
considered to be absent. In addition, when CCD documen- 
tation is generated, at least the patient demographics, the 
medication list coded with RxNORM or NCC, the allergy 
list coded with RxNORM, and the UNII information must 
be included. 
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3. Security Category 

Criteria related to security are composed of the following 
categories: Access control, Amendments, Audit, Authentica- 
tion, Backup/ Recovery, Data integrity Auditability, Docu- 
mentation, Manage clinical documentation, and Technical 
services. Of 43 criteria, 5 functions absent from BESTCare 
were confirmed as functional differences. Table 6 contains a 
detailed CCHIT criteria list showing the differences in func- 
tion. 

Currently, when BESTCare is used to view the full content 
of a finalized note (Amendments category, Table 6), all mod- 
ifications are logged; however, as BESTCare does not display 
the modifications on a screen in the finalized note format, 
this function was classified as absent and requiring supple- 
mentation. 

The ability to manage logs and auditable events was also 
implemented in BESTCare, but it is not possible to include 
or exclude auditable events by setting the program param- 
eters or to export time stamps in a coordinated universal 
time (UTC) standard time (i.e., 1994-ll-05T13:15:30-05:00) 
format. 

As the procedures to scan the system and installation me- 
dia for well-known malware (Technical Services category) 
were not documented, the related functions were mistakenly 
classified as absent. Furthermore, standards-based encryp- 
tion was required (but not currently available) when storing 



Table 5. Interoperability criteria and the inclusion of EHR (BESTCare) functions at Seoul National University Bundang Hospital 



Category 


Interoperability 


Yes No (type) 


Laboratory 


Receive and store general laboratory results using the HL7 v.2.5.1 ORU 
message standard 


I 


Medications/ ePrescribing 


Send an electronic prescription to the pharmacy 
Respond to a request for a refill sent from a pharmacy 
Send a query to verify prescription drug insurance eligibility 
Capture and display formulary information from the pharmacy or 

pharmacy benefits manager (PBM) 
Send aquery for medication history to PBM or pharmacy to capture and 

display medication list from the EHR 


5(P) 


Clinical documentation 


Display HITSP C32/CCD documents and file (e.g.: patient demographics, 

medication list, medication allergy list) 
Generate and format patient summary documents by HITSP C32 (v2.3 or 

v2.5) 


2(F) 


Total 




1 7 


F and P mean differences in functions and government policies. 

ORU: observation result, HITSP: Healthcare Information Technology Standards Panel, CCD: Continuity of Care Document, EHR: 
electronic health record. 
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Table 6. Criteria among the security category that is not included in the BESTCare 


Category/Criteria 


No. of criteria 


Amendments 


1 


View the full contents of a finalized note, including both the note and any finalized modifications to that 
note, such as corrections, clarifications, addenda, etc. 


Audit 


2 


Allow an authorized administrator to set the inclusion or exclusion of auditable events 
Format to export recorded time stamps using coordinated universal time based on ISO 8601 


Technical services 

The software used to install and update the system, independent of the mode or method of conveyance, 

shall be certified free of malevolent software ("malware"). 
When storing protected health information (PHI) on any device intended to be portable/removable, 

the system shall support use of a standards-based encrypted format using triple-data encryption standard 

(3-DES), the advanced encryption standard (AES), or their successors. 


2 


Total 


5 



protected health information (PHI) on a portable device. 

IV. Discussion 

EHRs must satisfy the qualification criteria for certification 
because they support clinical decision-making, input physi- 
cians' orders, and provide electronic healthcare information 
from other sources, capturing and querying data related, for 
example, to healthcare quality. Providers must also prove 
"meaningful use," including the electronic exchange of infor- 
mation to improve the quality and coordination of treatment 
via electronic prescriptions and quality-measure reports [14]. 
Furthermore, products with an emphasis on approaches such 
as usefulness, interoperability, and affordability should be 
used and reviewed in the market to allow decision-making. 

A certification system is a way to improve the quality of 
EHR products and secure interoperability, but is not a legal 
sanction. The participation of stakeholder groups and the 
opportunity for hearings must be conducted through a stan- 
dardized, transparent, and objective process that involves 
input from stakeholders. Additionally, a testing process that 
reflects the reality of use should be developed. It is necessary 
to select certification criteria based on the CCHIT certifica- 
tion process (such as EHR functionality, interoperability, 
and security) that are appropriate for Korean society and 
to develop a test process that reflects the needs of Korean 
health-care providers. For the healthcare information system 
in Korea to secure global competitiveness and dominate the 
international healthcare market, it must be equipped with a 
healthcare information system platform and solutions that 
meet international standards. Standardized healthcare in- 



formation technology is necessary to developing healthcare 
information services that ensure convenience, efficiency, and 
interoperability. In addition, to provide safe medical services 
and minimize users' reluctance or resistance, it is necessary 
to maintain the security and confidentiality of the healthcare 
data as well as to devote technical, legal, and ethical resourc- 
es to the protection of privacy. 

Through this study, we recognized the development of a 
CDSS that emphasizes patient-centered service in healthcare 
and identified the major EHR function that supports patients 
in prevention-related decision-making. CDSS ensures the in- 
teroperability of EHRs in addition to a common infrastruc- 
ture of healthcare information. Through the development of 
standards, services, and tools required for operation, CDSS 
will protect patients' interests and rights via improvements 
to patient safety and the prevention of medical malpractice, 
as well as by preventing the waste of resources [15-18]. 

Furthermore, the consideration of a wider variety of CDSS 
in the next global EHR system may yield a medical service 
that more faithfully tends towards care based on practice 
guidelines, with patient-centered care that contributes to 
enhanced patient surveillance and monitoring and reduced 
medication errors. Such a service is expected to significantly 
benefit the nation and the public interest, enhancing patient 
safety and management of service quality, as well as saving 
unnecessary medical expenses. 
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